83 research outputs found
Measuring Thematic Fit with Distributional Feature Overlap
In this paper, we introduce a new distributional method for modeling
predicate-argument thematic fit judgments. We use a syntax-based DSM to build a
prototypical representation of verb-specific roles: for every verb, we extract
the most salient second order contexts for each of its roles (i.e. the most
salient dimensions of typical role fillers), and then we compute thematic fit
as a weighted overlap between the top features of candidate fillers and role
prototypes. Our experiments show that our method consistently outperforms a
baseline re-implementing a state-of-the-art system, and achieves better or
comparable results to those reported in the literature for the other
unsupervised systems. Moreover, it provides an explicit representation of the
features characterizing verb-specific semantic roles.Comment: 9 pages, 2 figures, 5 tables, EMNLP, 2017, thematic fit, selectional
preference, semantic role, DSMs, Distributional Semantic Models, Vector Space
Models, VSMs, cosine, APSyn, similarity, prototyp
Is Structure Necessary for Modeling Argument Expectations in Distributional Semantics?
Despite the number of NLP studies dedicated to thematic fit estimation,
little attention has been paid to the related task of composing and updating
verb argument expectations. The few exceptions have mostly modeled this
phenomenon with structured distributional models, implicitly assuming a
similarly structured representation of events. Recent experimental evidence,
however, suggests that human processing system could also exploit an
unstructured "bag-of-arguments" type of event representation to predict
upcoming input. In this paper, we re-implement a traditional structured model
and adapt it to compare the different hypotheses concerning the degree of
structure in our event knowledge, evaluating their relative performance in the
task of the argument expectations update.Comment: conference paper, IWC
What a Nerd! Beating Students and Vector Cosine in the ESL and TOEFL Datasets
In this paper, we claim that Vector Cosine, which is generally considered one
of the most efficient unsupervised measures for identifying word similarity in
Vector Space Models, can be outperformed by a completely unsupervised measure
that evaluates the extent of the intersection among the most associated
contexts of two target words, weighting such intersection according to the rank
of the shared contexts in the dependency ranked lists. This claim comes from
the hypothesis that similar words do not simply occur in similar contexts, but
they share a larger portion of their most relevant contexts compared to other
related words. To prove it, we describe and evaluate APSyn, a variant of
Average Precision that, independently of the adopted parameters, outperforms
the Vector Cosine and the co-occurrence on the ESL and TOEFL test sets. In the
best setting, APSyn reaches 0.73 accuracy on the ESL dataset and 0.70 accuracy
in the TOEFL dataset, beating therefore the non-English US college applicants
(whose average, as reported in the literature, is 64.50%) and several
state-of-the-art approaches.Comment: in LREC 201
Unsupervised Measure of Word Similarity: How to Outperform Co-occurrence and Vector Cosine in VSMs
In this paper, we claim that vector cosine, which is generally considered
among the most efficient unsupervised measures for identifying word similarity
in Vector Space Models, can be outperformed by an unsupervised measure that
calculates the extent of the intersection among the most mutually dependent
contexts of the target words. To prove it, we describe and evaluate APSyn, a
variant of the Average Precision that, without any optimization, outperforms
the vector cosine and the co-occurrence on the standard ESL test set, with an
improvement ranging between +9.00% and +17.98%, depending on the number of
chosen top contexts.Comment: in AAAI 2016. arXiv admin note: substantial text overlap with
arXiv:1603.0870
The CogALex-V Shared Task on the Corpus-Based Identification of Semantic Relations
The shared task of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex-V) aims
at providing a common benchmark for testing current corpus-based methods for the identifica-
tion of lexical semantic relations (
synonymy
,
antonymy
,
hypernymy
,
part-whole meronymy
) and
at gaining a better understanding of their respective strengths and weaknesses. The shared task
uses a challenging dataset extracted from EVALution 1.0 (Santus et al., 2015b), which contains
word pairs holding the above-mentioned relations as well as semantically unrelated control items
(
random
). The task is split into two subtasks: (i) identification of related word pairs vs. unre-
lated ones; (ii) classification of the word pairs according to their semantic relation. This paper
describes the subtasks, the dataset, the evaluation metrics, the seven participating systems and
their results. The best performing system in subtask 1 is GHHH (
F
1
= 0
.
790
), while the best
system in subtask 2 is LexNet (
F
1
= 0
.
445
). The dataset and the task description are available at
https://sites.google.com/site/cogalex2016/home/shared-task
Taking antonymy mask off in vector space
Automatic detection of antonymy is an important task in Natural Language Processing (NLP) for Information Retrieval (IR), Ontology Learning (OL) and many other semantic applications. However, current unsupervised approaches to antonymy detection are still not fully effective because they cannot discriminate antonyms from synonyms. In this paper, we introduce APAnt, a new Average-Precision-based measure for the unsupervised discrimination of antonymy from synonymy using Distributional Semantic Models (DSMs). APAnt makes use of Average Precision to estimate the extent and salience of the intersection among the most descriptive contexts of two target words. Evaluation shows that the proposed method is able to distinguish antonyms and synonyms with high accuracy across different parts of speech, including nouns, adjectives and verbs. APAnt outperforms the vector cosine and a baseline model implementing the co-occurrence hypothesis
Extensive Evaluation of Transformer-based Architectures for Adverse Drug Events Extraction
Adverse Event (ADE) extraction is one of the core tasks in digital
pharmacovigilance, especially when applied to informal texts. This task has
been addressed by the Natural Language Processing community using large
pre-trained language models, such as BERT. Despite the great number of
Transformer-based architectures used in the literature, it is unclear which of
them has better performances and why. Therefore, in this paper we perform an
extensive evaluation and analysis of 19 Transformer-based models for ADE
extraction on informal texts. We compare the performance of all the considered
models on two datasets with increasing levels of informality (forums posts and
tweets). We also combine the purely Transformer-based models with two
commonly-used additional processing layers (CRF and LSTM), and analyze their
effect on the models performance. Furthermore, we use a well-established
feature importance technique (SHAP) to correlate the performance of the models
with a set of features that describe them: model category (AutoEncoding,
AutoRegressive, Text-to-Text), pretraining domain, training from scratch, and
model size in number of parameters. At the end of our analyses, we identify a
list of take-home messages that can be derived from the experimental data
AILAB-Udine@SMM4H 22: Limits of Transformers and BERT Ensembles
This paper describes the models developed by the AILAB-Udine team for the
SMM4H 22 Shared Task. We explored the limits of Transformer based models on
text classification, entity extraction and entity normalization, tackling Tasks
1, 2, 5, 6 and 10. The main take-aways we got from participating in different
tasks are: the overwhelming positive effects of combining different
architectures when using ensemble learning, and the great potential of
generative models for term normalization.Comment: Shared Task, SMM4H, Transformer
- …